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IN THE CLAIMS: 

The text of all pending claims are set forth below. Cancelled and withdrawn claims are 
indicated with claim number and status only. The claims as listed below show added text with 
underlining and deleted text with str i k e through . The status of each claim is indicated with one of 
(original), (currently amended), (previously presented), (cancelled), (withdrawn), (new), 
(previously added), (reinstated - formerly claim #), (previously reinstated), (re-presented - 
formerly dependent claim #) or, (previously re-presented). 

Please AMEND the claims in accordance with the following: 

1 . (CURRENTLY AMENDED) A robot system having an image processing function 
for determining orientation, or orientation and position of a robot operation on an obj e ct of 
dotoction among one of a plurality of objects, the system comprising: 

a robot; 

a first image capturing device capturing image data of the plurality of objects containing 
respective images of the objects : 

a memory storing reference models, each comprising an image of a reference object 
captured by said image capturing device in a different direction, and for each reference model 
storing information of the capturing direction of its associated image and information of an 
orientation of the robot with respect to the reference object, the information of the capturing 
direction representing a rotational posture of the reference object relative to the robot, said 
reference object being one of the plurality of objects o r an object having a shape substantia ll y 
identical to that of one of the plurality of objects; and 

a processor to perform matching proc e ssing on the image data containing images of the 
plurality of objects captured by said first image capturing device with each of said reference 
mod e l imag e s models successively and a work i ng i mag e of th e p l ura l ity of ob je cts (captur e d by 
sa i d first i mag e captur i ng dev i c e ) to select one object having an image ef- matched with one of 
the- said reference models that match e s th e obj e ct of d e t e ction in th e work i ng i mag e, and to 
determine the-orientation, or the-orientation and position of the robot operation to bo perform e d 
on th e obj e ct of d e t e ct i on, th e d e t e rm i n i ng based on the s ele ct e d image of the r e f e r e nc e 
selected one object, based on said one reference model and the information of its associated 
capturing direction, and based on the information of the orientation of the robot operation with 
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respect to the reference object that is associated with said one reference model. 

2. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 1 , wherein said reference models are obtained from a part of the 
image data of the reference object. 

3. (ORIGINAL) A robot system having an image processing function according to 
claim 1, wherein said reference models are obtained by processing the image data of the 
reference object. 

4. (ORIGINAL) A robot system having an image processing function according to 
claim 1 , wherein said first image capturing device comprises a camera for capturing two- 
dimensional image data. 

5. (ORIGINAL) A robot system having an image processing function according to 
claim 4, wherein said image data of the reference object are captured by said camera from a 
predetermined distance. 

6. (CURRENTLY AMENDED) A robot system having an image processing function 
according to claim 1 , further comprising: 

a second image capturing device; wherein 

said robot situates said second image data capturing device to have said determined 
orientation or to have said determined orientation and said determined position with respect to 
the selected one object, and wherein 

said processor processes second image data captured by said second image capturing 
device to detect position and/o r rotational posture of the selected one object with respect to said 
second image data capturing device. 

7. (CURRENTLY AMENDED) A robot system having an image processing function 
according to claim 1 further comprising: 

a second image capturing device for obtaining three-dimensional position; wherein 
said robot situates said second image data capturing device to have said determined 
orientation or to have said determined orientation and said determined position with respect to 
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the selected one object, so that said second image data capturing device is directed to a 
characterizing portion of the object; and wherein 

said processor detects three-dimensional position and/or posture of the selected one 
object based on three-dimensional position of said characterizing portion obtained by said 
second image capturing device. 

8. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 6, wherein said first image data capturing device is used as said 
second image data capturing device. 

9. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 6, wherein said second image capturing device comprises a three- 
dimensional visual sensor of spot-light scanning type capable of measuring distance between 
the sensor and an object. 

10. (CURRENTLY AMENDED) A robot system having an image processing function 
according to claim 6, wherein said second image data capturing device comprises a structured- 
light unit for irradiating a structured light on an- the selected object and capturing an image of the 
object including the irradiated light on the object. 

11. (CURRENTLY AMENDED) A robot system having an image processing function 
according to claim 7, wherein said robot operation is an operation of picking up at le ast the 
selected one object from a- the plurality of objects , some of which are overlapped with each 
other 

12. (CURRENTLY AMENDED) A robot system having an image processing function 
for determining orientation, or orientation and position of a robot operation on an obj e ct of 
d e t e ction among one of a plurality of objects of plural kinds, where the orientation of the 
operation corresponds to a determined orientation of the one object, where the determined 
orientation is a rotational posture of the one object the system comprising: 

a robot; 

a first image capturing device capturing image data of the plurality of objects containing 
respective images of the objects ; 
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a memory storing reference models, each comprising images of each of different kinds of 
reference objects corresponding to images captured by said first image capturing device, and 
storing indicia of the kinds respectively associated with said reference models, and information 
of a different orientation of the robot with respect to each of the different images of the reference 
object of each kind, where the captured information of orientation comprises information of a 
rotational posture of the reference object relative to the robot, each of said reference objects 
being one of the kinds of the plurality of objects or having a shape substantially identical te 
thereto; and 

a processor to perform matching proc e ss i ng on the image data containing images of the 
plurality of objects captured by said first image capturing device with each of said reference 
mod el i mag e s models successfully and a work i ng imag e of th e p l ura l ity of obj e cts of p l ura l k i nds 
(captur e d by sa i d first i mag e capturing d e vic e )' to select one object having an image of- matched 
with one of said kinds of the reference models that match e s th e obj e ct of d e t e ction in th e 
work i ng i mag e, and to determine tbe-orientation, or the-orientation and position of the robot 
operation to b e p e rformed on th e obj e ct of d e t e ct i on , the determining based on the sel e ct e d 
image of the selected one object , based on said one reference model, based on the indicia of 
the kind associated with said one reference model and the information of the orientation of the 
robot operation with respect to the reference object associated with said one reference model of 
said one kind. 

13. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 12, wherein said reference models are obtained from a part of the 
image data of the reference object. 

14. (ORIGINAL) A robot system having an image processing function according to 
claim 12, wherein said reference models are obtained by processing the image data of the 
reference object. 

15. (ORIGINAL) A robot system having an image processing function according to 
claim 12, wherein said first image capturing device comprises a camera for capturing two- 
dimensional image data. 
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16. (ORIGINAL) A robot system having an image processing function according to 
claim 15, wherein said image data of the reference object are captured by said camera from a 
predetermined distance. 

17. (CURRENTLY AMENDED) A robot system having an image processing function 
according to claim 12, further comprising a second image capturing device, wherein 

said robot situates said second image data capturing device to have said determined 
orientation or to have said determined orientation and said determined position with respect to 
the object, and wherein 

said processor processes second image data captured by said second image capturing 
device to detect position and/or posture of the selected one object with respect to said second 
image data capturing device. 

18. (CURRENTLY AMENDED) A robot system having an image processing function 
according to claim 12, further comprising: 

a second image capturing device for obtaining three-dimensional position; wherein 
said robot situates said second image data capturing device to have said determined 

orientation or to have said determined orientation and said determined position with respect to 

the selected one object, so that said second image data capturing device is directed to a 

characterizing portion of the object; and wherein 

said processor detects three-dimensional position and/or posture of the selected one 

object based on three-dimensional position of said characterizing portion obtained by said 

second image capturing device. 

19. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 17, wherein said first image data capturing device is used as said 
second image data capturing device. 

20. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 17, wherein said second image capturing device comprises a three- 
dimensional visual sensor of spot-light scanning type capable of measuring distance between 
the sensor and an object. 
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21 . (CURRENTLY AMENDED) A robot system having an image processing function 
according to claim 17, wherein said second image data capturing device comprises a structured- 
light unit for irradiating a structured light on aft- the selected one object and capturing an image 
of the object including the irradiated light on the object. 

22. (CURRENTLY AMENDED) A robot system having an image processing function 
according to claim 18, wherein said robot operation is an operation of picking up a t le ast the 
selected one object from a- the plurality of objects , some of which are overlapped with each 
other. 

23. (CURRENTLY AMENDED) A method for automatically determining an 
arrangement of a workpiece relative to a robot , where the determined arrangement comprises at 
least rotational posture arrangement of the workpiece relative to the robot , the method 
comprising: 

storing reference images corresponding to images of the workpiece or an object so 
shaped (workpiece/object) and reference arrangement information indicating arrangements of 
the robot and workpiece/object relative to each other when the images were captured , the 
reference arrangements comprising rotational arrangements of the workpiece relative to the 
robot ; 

from a known arrangement of the robot, capturing a working image of the workpiece 
among a plurality of randomly arranged workpieces with an imaging device; 

finding one of the reference images that has a c l os e st matches te-the workpiece in the 
working image; and 

determining a known an arrangement of the robot relative to the workpiece based on 
information indicating the known arrangement of the robot, and based on the reference 
arrangement information corresponding to the found reference image , where the determined 
arrangement comprises rotational arrangement of the workpiece relative to the robot . 

24. (PREVIOUSLY PRESENTED) A method according to claim 23, wherein 
reference images and reference arrangement information is obtained for workpieces/objects of 
different shapes, and wherein the finding comprises first determining that a reference image of 
one of the different shapes matches the working image of the workpiece, and then finding one 
reference image of the shape that best matches the working image. 
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25. (PREVIOUSLY PRESENTED) A method according to claim 23, wherein the robot 
is used to capture the reference images, and wherein the reference arrangement information 
represents arrangements of the robot when capturing the reference images. 

26. (CURRENTLY AMENDED) A method according to claim 23, wherein a second 
imaging device is affixed to the am -robot and is used to determine additional arrangement 
information used to determine the known arrangement of the robot relative to the workpiece. 

27. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 7, wherein said first image data capturing device is used as said 
second image data capturing device. 

28. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 7, wherein said second image capturing device comprises a three- 
dimensional visual sensor of spot-light scanning type capable of measuring distance between 
the sensor and an object. 

29. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 7, wherein said second image data capturing device comprises a 
structured-light unit for irradiating a structured light on an object and capturing an image of the 
object including the irradiated light on the object. 

30. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 18, wherein said first image data capturing device is used as said 
second image data capturing device. 

31 . (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 18, wherein said second image capturing device comprises a three- 
dimensional visual sensor of spot-light scanning type capable of measuring distance between 
the sensor and an object. 
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32. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 18, wherein said second image data capturing device comprises a 
structured-light unit for irradiating a structured light on an object and capturing an image of the 
object including the irradiated light on the object. 
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